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ARE CHARTER SCHOOLS EFFECTIVE? 

Background for the Study 

Within the context of continual dissatisfaction with American 
public schools, charter school proponents view charter schools as a viable 
reform mechanism. At the heart of their reform potential is a degree of 
exemption from state bureaucratic and regulatory red tape which tradi- 
tional public schools do not have (Sarason, 1998). This exemption would 
allow schools to tailor their programs and to experiment to attract and bet- 
ter serve students. As of spring 2003, forty states in the United States have 
enacted charter school legislation; 2,500 charter schools are operating. 
The numbers of schools established in each of the forty states varies from 
none in Iowa, New Hampshire, and Tennessee to well over two hundred in 
Arizona (Center for Education Reform, 2003). 

The degree of freedom and autonomy afforded charter schools 
varies from state to state however. The Center for Education Reform con- 
ducts yearly rankings of state charter school laws based on ten criteria 
concerning charter school operations and their relationship to their char- 
tering entity. The score for each state’s charter school law is then assigned 
a letter grade and a classification as being either strong (encouraging the 
development of charter schools) or weak (discouraging the development 
of charter schools) (Center for Education Reform, 2003). In this way, 
states are categorized regarding the strength of their charter school 
statutes. 

A strong charter school law encourages the development of char- 
ter schools within a state by: (a) permitting an unlimited or substantial 
number of charter schools; (b) allowing a number of entities in addition to 
the local school board to authorize charter schools; (c) permitting a variety 
of individuals and groups both inside and outside the existing public 
school system to start charter schools; (d) permitting new schools to start 
up from scratch; (e) permitting charter schools to be started without prov- 
ing specified levels of local support; (f) providing automatic blanket 
waivers from most or all state education laws and regulations; (g) permit- 
ting charter schools to be independent legal entities; (h) guaranteeing 100 
percent of per-pupil state funding to charter schools; (i) permitting charter 
schools to control their funds; and (j) giving charter schools complete con- 
trol over personnel decisions (Center for Education Reform, 2003). 

On the other hand, a weak charter school law controls the devel- 
opment of charter schools in a state by: (a) restricting the number of char- 
ter schools; (b) limiting the number of entities, such as the local school 
board, that can authorize charter schools; (c) limiting the variety of indi- 
viduals and groups who can start charter schools; (d) permitting only 
existing public schools to be converted into charter schools; (e) requiring 
the demonstration of specified levels of support prior to the establishment 
of the charter school; (f) requiring or not allowing charter schools to apply 
to the chartering body for waivers from specific education laws and regu- 
lations; (g) requiring charter schools to be under the governance of the 
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local school district; (h) setting funding at levels less than 100 percent of 
per pupil funding in the state or by requiring negotiations for funding with 
the local school district; (i) allowing funds to only be disbursed through 
the local school district; and (j) requiring adherence to the local school 
district’s collective bargaining agreements or work rules (Center for Edu- 
cation Reform, 2003). 

With these degrees of freedom from red tape comes responsibili- 
ty. Are charter schools effective? One means by which to address this 
question is the u t ili z ation of the effective schools research. In her 2002 
Phi Delta Kappan article, Barbara Taylor finds that the effective schools 
process is “alive and well” (p. 375). She notes that the process and corre- 
lates of effective schools have been accepted as one of the approved com- 
prehensive school reform models in the U.S. Department of Education’s 
Comprehensive School Reform initiative. The process and correlates have 
also become the foundation for several school reform initiatives around 
the nation, including Phi Delta Kappa’s National Center for Effective 
Schools, the California Center for Effective Schools, the Association for 
Effective Schools, and the National Alliance for Effective Schools. 

Ron Edmonds’ work with the New York City schools was one of 
the earliest attempts at applying the effective schools research to school 
improvement at the local level, and it provided a model for others to fol- 
low. Subsequently, a number of school improvement programs were 
implemented in cities around the country and the effective schools model 
was adopted by most state departments of education. At the same time, 
characteristics of effective schools began to evolve. While there were dif- 
ferences in details from one researcher to the next, common elements in 
all effective schools research included: a student achievement focus, an 
emphasis on all students, and a goal of mastery of basic skills (Mace- 
Matluck, 1987). Mace-Maduck (1986) offered a composite of definitions 
found in the literature: “An effective school is one in which the conditions 
are such that student achievement data show that all students evidence an 
acceptable minimum mastery of those essential skills that are prerequisite 
to success at the next level of schooling” (p. 5). 

In three well-known summarizations (1979a, 1979b, 1981), Ron 
Edmonds maintains that there are five correlates of effective schools: (a) 
The leadership of the principal is characterized by substantial attention to 
the quality of instruction; (b) there is a pervasive and broadly understood 
instructional focus; (c) an orderly, safe climate exists that is conducive to 
teaching and learning; (d) teacher behaviors convey the expectation that 
all students are to obtain at least minimum mastery; and (e) pupil achieve- 
ment is used as the measure for program evaluation. Edmonds used the 
term “correlate” arguing that each of the above ingredients is related to the 
others, that they are interactive, and that they are all present in effective 
schools (Mace-Matluck, 1987). 

Other reviewers of the effective schools literature did not always 
find the same set of features to be characteristics of effective schools. This 
resulted in the compilation of somewhat different lists of features typical- 
ly categorized as characteristics or variables. Most often-cited are reviews 
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by Tomlinson (1980), Austin (1979, 1981), Duckett, Park, and Clark 
(1980), and Purkey and Smith (1983) (Mace-Matluck, 1987). 

Brookover and Lezotte (1979), Gauthier (1982), and Villanova 
(1984), among others, have honed the seven effective school correlates 
that are most widely accepted today (Levine & Lezotte, 1990). They are: 

(a) clearly stated and focused school mission; (b) high expectations for 
students, teachers, and administrators; (c) safe and orderly climate for 
learning; (d) instructional leadership by all administrators and staff mem- 
bers; (e) opportunity to learn and time-on-task; (f) frequent monitoring of 
student progress; and (g) positive home/school relations (Taylor, 2002, p. 
377). The premise is that these correlates can effectively guide school 
improvement and reform. 

Purposes of the Study 

The purposes of this study were (a) to compare the perceptions of 
charter school principals regarding the extent to which the effective 
school correlates have been implemented in their respective schools and 

(b) to compare the responses of charter school principals in states with 
strong charter school laws to the responses of charter school principals in 
states with weak charter school laws to determine if a relationship exists 
between the strength of a state’s charter school law and the extent to 
which the effective school correlates have been implemented. Charter 
school principals were chosen to respond to the survey based on the over- 
arching nature of the typical school principal’s job, placing them in a bet- 
ter position to be knowledgeable about all facets of the school and 
operations than classroom teachers or other school employees. 

The statistical hypotheses used in this study postulate equality 
among sub-population means. When asking the question “Do charter 
school principals in states with weak charter school laws and charter 
school principals in states with strong charter school laws differ with 
respect to the extent that they perceive their schools have implemented the 
effective school correlates?,” the following statistical hypothesis was test- 
ed: Hq: Pi = P 2 - The mean for the distribution of responses for principals 
of charter schools in states with weak charter school laws is represented 
by pp the mean for the distribution of responses for principals of charter 
schools in states with strong charter school laws is represented by P 2 ; and 
the mean for the distribution of responses for all principals of charter 
schools is represented by Hq. 


Methods 

Sample and Sampling Procedures 

For the purposes of this investigation, the subjects were chosen 
from the principals or directors of all the nation’s charter schools. In spite 
of the probable respondent bias, the researchers chose these subjects 
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because their positional perspective offers the most inclusive, general 
understanding of the schools’ implementation of the Effective School 
Correlates. 

The researchers gathered data from all forty states with charter 
schools operating during spring 2003. All 255 charter schools located in 
states with weak charter school laws, as determined by the rankings of the 
Center for Education Reform (2003), were selected for participation in the 
study. An equal number of charter schools were selected in a simple ran- 
dom sample from the 2441 charter schools located in states with strong 
charter school laws. 

A total of 166 participants responded to the survey; 88 from prin- 
cipals of charter schools in states with weak charter school laws and 78 
from principals of charter schools in states with strong charter school 
laws. 

Measures 

Numerous surveys in the professional literature were considered 
as possible instruments for this study. Due to having a Cronbach alpha 
coefficient of .9051 and a history of successful utilization, the Effective 
School Survey (Holifield, 2002) was utilized. The instrument contained 
seven sections with each section corresponding to one of the seven effec- 
tive school correlates: Instructional Eocus (five indicators). High Expecta- 
tions (four indicators). School Climate (four indicators). Instructional 
Eeadership (six indicators), Time-on-Task (six indicators). Monitoring of 
Student Progress (three indicators), and School-Community Relations 
(three indicators).^ Study participants noted the extent to which each cor- 
relate’s indicators were implemented in their schools: 4 = Met very well, 3 
= Generally met, 2 = Not met very well, 1 = Not met at all, 0 = Don’t 
know. In determining the classification of the mean of means for each cor- 
relate and for the study as a whole, the following values were used: 3.50 
and above was classified as “met very well,” 2.50-3.49 was classified as 
“generally met,” 1.50-2.49 was classified as “not met very well,” 
0.50-1.49 was classified as “not met at all,” 0.49 and below was classified 
as “don’t know.” 

Some survey items could be deemed “double-barreled” because 
they contain two or more variables that could arguably cause multiple 
interpretations of the items and produce inconsistent response patterns. 
The researchers believed this characteristic does not seriously detract 
from the validity of these items but more importantly does not detract 
from the study’s conclusions regarding the relationship between charter 
schools and implementation of the effective school correlates. As previ- 
ously noted, the survey has a Cronbach alpha coefficient of .9051 signify- 
ing internal consistency and suggesting a lack of inconsistent response 
patterns due to ambiguity. More importantly, the multiple embedded vari- 
ables within an item are cognitively bundled as concomitant elements of a 
construct that would be very familiar in the everyday world of practice. 
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Data Analysis 

SPSS statistical software was used to employ an Independent 
Samples t-test to determine the significant differences, if any, between the 
independent variables (schools in states with strong and weak charter 
school laws) and each of the study’s dependent variables, the survey 
items. Further data analysis of each of the effective school correlates was 
conducted by using subgroup mean scores. The mean scores for the entire 
survey were used to draw conclusions relative to the study’s hypothesis. 
Levene’s Test for Equality of Variances was used to determine differences 
in variation in the spread of responses between the two groups. The p < 
.05 level of probability was used to note significance. 

Findings 

The findings of the study are presented in order by correlate. The 
table for each correlate contains the correlate itself (in parentheses after 
the title), the actual indicators from the survey instrument, the means for 
each sample group, and the mean of means for the correlate. 

Instructional Focus 

The Instructional Focus section of the survey (see Table 1) con- 
tained five indicators concerning the school’s purposes and goals. Weak- 
state principals reported that indicators 2 (variety of teaching models 
used) and 5 (periodic review of the school’s mission) were met very well 
while strong-state principals also reported indicator 5 as met very well. 
The mean of means for all indicators in the Instructional Focus Correlate 
was 3.30 for weak states and 3.23 for strong states indicating that both 
groups reported this correlate as generally met. There were no statistically 
significant differences noted. 

Table 1 

Results for Instructional Focus Correlate (A clear instructional focus 
represents the school’s guiding purposes and goals) 

Weak- Strong- 

state state 

Indicators means means 


1. Learning goals and objectives, instructional 3.42 3.46 

strategies, activities, and resources, and student 

assessments are aligned. 

2. Teachers use a variety of teaching models to 3.61 3.49 

address a variety of learning styles. 

(continued) 
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Table 1 (continued) 

Weak- Strong- 

state state 

Indicators means means 

3. Vertical and horizontal curricular alignment 3.07 2.94 

across grade levels and subjects. 

4. Extra-curricular activities are guided by specific 2.73 2.68 

objectives and learner outcomes that contribute 

to accomplishing the school’s mission. 

5. The school’s mission (i.e. goals and objectives) 3.65 3.58 

is periodically reviewed. 

Mean of means 3.30 3.23 


High Expectations 

The High Expectations section of the survey (see Table 2) con- 
tained four indicators concerning the role of the school in removing obsta- 
cles to learning and in recognizing and rewarding strong performance. 
There were no statistically significant differences noted in the responses 
from weak-state principals and strong-state principals, with the responses 
indicating that these indicators were at least generally met. Weak-state 
principals reported that indicator 7, concerning meeting the diverse needs 
of students, was met very well. The mean of means for the High Expecta- 
tions Correlate was 3.19 for weak states and 3.22 for strong states indicat- 
ing that both groups reported this correlate to be generally met. 

Table 2 


Results for High Expectations Correlate (All students are expected to 
attain mastery of essential school skills) 


Indicators 

Weak- 

state 

means 

Strong- 

state 

means 

6. Excellence in achievement and behavior inclusive 

2.94 

3.12 


of students, teachers, administrators, and staff is 
recognized via awards that have explicit require- 
ments. 

7. Individual student differences in interest, 3.52 3.46 

probable needs, special aptitudes, and ability are 
addressed, especially via enrichment and 
remediation activities. 

(continued) 
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Table 2 (continued) 


Indicators 

Weak- 

state 

means 

Strong- 

state 

means 

8. Established policies and procedures contribute to 
teacher efficacy regarding utilization of social 
services. 

2.95 

2.87 

9. Mastery criteria regarding assignments are clearly 
communicated to students. 

3.36 

3.41 

Mean of means 

3.19 

3.22 


School Climate 

The School Climate section of the survey (see Table 3) contained 
four indicators dealing with establishing a school environment that supports 
academic performance. There were no statistically significant differences 
noted between the responses of weak-state principals and strong-state prin- 
cipals with the means for both groups indicating that indicators 10 (well- 
maintained physical facilities) and 11 (consistent discipline plans) were 
very well met, and that indicators 12 (classroom discipline plans) and 13 
(induction programs) were generally met. The mean of means for the 
School Climate Correlate was 3.38 for weak states and 3.41 for strong states 
indicating both groups generally met this correlate. 

Table 3 


Results for School Climate Correlate (The school has a purposeful, busi- 
nesslike atmosphere that is not oppressive) 


Indicators 

Weak- 

state 

means 

Strong- 

state 

means 

10. Physical facilities are well maintained, kept 
clean, and reasonably attractive. 

3.56 

3.63 

11. Discipline plans — established through collabor- 
ation with appropriate stakeholders — are commu- 
nicated and consistendy implemented. 

3.50 

3.53 

12. Individual classroom discipline plans operation- 
alize the building discipline plan, which in turn 
operationalizes the district discipline plan. 

3.27 

3.47 

13. Induction programs are implemented for new 
faculty and students. 

3.17 

3.01 

Mean of means 

3.38 

3.41 
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Instructional Leadership 

The Instructional Leadership section of the survey (see Table 4) 
contained six indicators concerning the role of leaders in the instructional 
program of the school. As with the previous correlates, there were no sta- 
tistically significant differences noted between the responses from the two 
groups with the means for both indicating that each indicator was at least 
generally met. Both groups reported that indicator 18 (Leader facilitation 
of in-service and mentoring) was met very well. The mean of means for 
the Instructional Leadership Correlate was 3.37 for weak states and 3.31 
for strong states indicating that both groups generally met this correlate. 

Table 4 


Results for Instructional Leadership Correlate (Leaders understand and 
apply the characteristics of instructional effectiveness in the management 
of the instructional program) 




Weak- 

Strong- 



state 

state 

Indicators 

means 

means 

14. 

Teacher supervision focuses on instruction and 
learning and results in thoughtful, constructive, 
research-based feedback. 

3.44 

3.31 

15. 

Data from multiple types of assessments are used 
to assess personnel. 

3.00 

3.03 

16. 

Supervisory visits to classrooms to observe 
instruction are frequent and focused on teachers’ 
instructional needs. 

3.34 

3.27 

17. 

Leaders ensure that improvement goals and 
objectives are collaboratively established, 
implemented, and assessed. 

3.44 

3.36 

18. 

Leaders facilitate in-service and mentoring 
activities that are aligned with school improve- 
ment goals. 

3.51 

3.54 

19. 

Leaders use established procedures and quality 
criteria for recruiting teachers. 

3.49 

3.32 


Mean of means 

3.37 

3.31 


Time-on-Task 

The Time-on-Task section of the survey included six indicators 
concerned with the school’s ability to provide unimpeded blocks of time 
for instruction and how teachers and students use that time. There were no 
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statistically significant differences noted for indicators 20-24 between the 
responses for weak and strong states. Means for both groups indicated that 
indicators 20 and 21 were generally met and indicators 22, 23, and 24 
were met very well. A statistically significant difference was noted in the 
responses to indicator 25 (extra help for students) at the p - .020 level of 
significance with the mean for weak states indicating the item was met 
very well and the mean for strong states indicating it was generally met. 
The mean of means for the Time-on-Task Correlate was 3.55 for weak 
states indicating this correlate was met very well and 3.43 for strong states 
indicating this correlate was generally met. 

Table 5 


Results for Time-on-Task Correlate (Multiple strategies ensure students 
are, for a high percentage of time, actively engaged in learning) 


Indicators 

Weak- 

state 

means 

Strong- 

state 

means 

20. The school uses a multi-faceted program to 
maintain a high level of student attendance. 

3.40 

3.21 

21. Teachers and administrators practice management 
and supervisory techniques that keep students 
on task and minimize disruptions. 

3.45 

3.45 

22. Students are engaged during the vast majority of 
class time. 

3.61 

3.62 

23. School events are scheduled to avoid disruption 
of learning time. 

3.58 

3.54 

24. Student pullouts for academic or non-academic 
purposes from regular classes are minimized. 

3.70 

3.54 

25. Students can get extra help from school outside 
regular hours. 

3.56 

3.23 

Mean of means 

3.55 

3.43 


Monitoring of Student Progress 

The survey addressed the correlate on Monitoring of Student 
Progress through three indicators of how assessment procedures are used 
to improve student achievement and the instructional program (see Table 
6). The means for both weak-state and strong-state principals indicated 
that Indicator 26 (multiple assessments) was met very well while Indica- 
tors 27 and 28 were generally met. The mean of means for the Monitoring 
of Student Progress Correlate was 3.39 for weak states and 3.40 for strong 
states indicating that both groups generally met this correlate. There were 
no statistically significant differences. 
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Table 6 


Results for Monitoring of Student Progress Correlate (A variety of 
student assessment procedures are used to improve individual student 
achievement as well as the instructional program) 


Indicators 

Weak- 

state 

means 

Strong- 

state 

means 

26. Data from multiple types of assessments are used 
to assess students’ academic progress. 

3.57 

3.59 

27. The school analyzes the performance and progress 
of specific groups of students to assure that all are 
learning well. 

3.28 

3.38 

28. Testing results are used directly to assess the 
instructional program, objectives, materials, and 
the behavior of personnel. 

3.31 

3.24 

Mean of means 

3.39 

3.40 


School-Community Relations 

The portion of the survey devoted to the School-Community 
Relations Correlate (see Table 7) contained three indicators dealing with 
how the school communicates with and involves parents in the education 
of their children. The means for both weak-state and strong-state princi- 
pals indicate that these three indicators were all generally met. The mean 
of means for the School-Community Relations Correlate was 3.35 for 
weak states and 3.22 for strong states indicating that both groups general- 
ly met this correlate. There were no statistically significant differences. 

Table 7 


Results for School-Community Relations Correlate (Parents are given 
multiple opportunities to play important roles in helping the school to 
achieve its mission) 




Weak- 

Strong- 



state 

state 

Indicators 

means 

means 

29. 

A planned school-community relations program 
is continually utilized by the school. 

3.25 

3.01 

30. 

Parents have various options for becoming 

3.36 

3.19 


involved in making decisions regarding schooling, 
especially in the development and support of 
the instructional program. 

(continued) 
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Table 7 (continued) 



Weak- 

Strong- 


state 

state 

Indicators 

means 

means 

31. Faculty provide parents with information and 

3.45 

3.46 

techniques for helping students learn. 



Mean of means 

3.35 

3.22 


Summary of Effective Schools Correlates 

The mean of means for all correlates was 3.36 for weak states and 
3.32 for strong states indicating that both groups generally met all effec- 
tive school correlates. 


Conclusions 

Several conclusions can be made based on the reported percep- 
tions of the study’s respondents. The null hypothesis cannot be rejected. 
The Effective Schools Correlates are, at the least, “generally met” in char- 
ter schools in both strong and weak states. In spite of the greater degree of 
freedom from bureaucratic red tape, the charter schools have not, in gen- 
eral, implemented the correlate indicators of this study’s survey to the 
“met well” level. However, most of these schools are, most of the time, 
utilizing the behaviors implicit in this survey’s indicators. The researchers 
do not know if this level of implementation would be significantly differ- 
ent from non-charter public schools. 

Nevertheless, the relative strength or weakness of a state’s charter 
school law appears, at least from the perceptions of charter school princi- 
pals, to have little, if any, impact on the extent to which the effective 
schools correlates are implemented in their schools. In fact, the only sig- 
nificant difference in perceptions pertained to Indicator 25 of the Time- 
on-Task Correlate regarding the availability of extra help for students 
from the school outside regular hours; respondents from weak states met 
the condition of the descriptor very well while strong-state respondents 
generally met the condition. Perhaps policymakers should note that the 
design and effectiveness of charter school programs and procedures may 
be independent of the degree of expansiveness or restrictiveness of the 
charter school laws. 

The lack of significant differences in responses from the two 
groups may result from several factors. Respondent bias, whether con- 
scious or unconscious, is a well-known limitation of self-reporting survey 
research (Gall, Borg, & Gall, 1996) and may have caused inflated 
responses from one or both groups. Also, the sample group, charter school 
principals, may be too narrow in scope and limited to a population that is 
likely to think along similar lines. 
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Recommendations 

The following are recommendations for further investigation: (a) 
Replicate this study utilizing the population of classroom teachers and stu- 
dents in charter schools located in both strong and weak charter school 
law states to determine if their perceptions differ from those of the sub- 
jects of this study; (b) conduct a qualitative study of the extent to which 
the Effective Schools Correlates have been implemented in charter 
schools using various stakeholders as the subjects; and (c) conduct a study 
to compare and contrast the extent to which non-charter and charter 
schools implement the effective school correlates. 

Note 

Tndicators for the Effective Schools Survey used in this research 
were developed from the following resources: Arkansas COE-Teacher 
Survey (North Central Association, n.d.); Characteristics of Effective 
Schools (Missouri Department of Elementary and Secondary Education, 

1985) ; The Effective Education Series (Colorado Department of Education, 

1986) ; An Administrator’s Guide for Evaluating Programs and Personnel 
(DeRoche, 1981); Kelwynn’s Staff Development for Effective Schools: 
Cadre Training Manual for School Improvement Teams (Human Synergis- 
tics, 1985); Educational Administration: Concepts and Practices (Eunen- 
burg & Ornstein, 1991); Ventures in Good Schooling (NASSP/NEA, 
1986); The Newark, New Jersey, School Effectiveness Questionnaire 
(Camden Middle School, 1985); The Development of a Survey of School 
Effectiveness (Ruhl, 1985); and Ready Schools (Shore, 1998). 
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